Ben Marwick, University of Washington
December 2014
“An article about a computational result is advertising, not scholarship. The actual scholarship is the full software environment, code and data, that produced the result.” - Claerbout and Karrenbach, Proceedings of the 62nd Annual International Meeting of the Society of Exploration Geophysics. 1992
“When we publish articles containing figures which were generated by computer, we also publish the complete software environment which generates the figures” - Buckheit & Donoho, Wavelab and Reproducible Research, 1995.
“The scholarship does not only consist of theorems and proofs but also (and perhaps even more important) of data, computer code and a runtime environment which provides readers with the possibility to reproduce all tables and figures in an article.” - Hothorn et al., 2009 Case studies in reproducibility
Technical
Cultural & personal
Peng 2011, Science 334(6060) pp. 1226-1227
Click-trails compromise clarity
Scripted analyses support scientific integrity
Using literate statistical programming
Using an open source programming language
Using an open document formatting language
Using version control
Using dynamic documents
Stodden (IASSIST 2010) sampled American academics registered at the Machine Learning conference NIPS (134 responses from 593 requests (23%). Red = communitarian norms, Blue = private incentives
Stodden (IASSIST 2010) sampled American academics registered at the Machine Learning conference NIPS (134 responses from 593 requests (23%). Red = communitarian norms, Blue = private incentives
An incentive to share data and code by acknowledging open practices with badges in publications. Currently used by Psychological Science
“Abandoning the habit of secrecy in favor of process transparency and peer review was the crucial step by which alchemy became chemistry.”
-Raymond, E. S., 2004, The art of UNIX programming: Addison-Wesley.
Presentation written in Markdown (R Presentation)
Compiled into HTML5 using RStudio
Source code hosting: https://github.com/benmarwick/INCF-Sharing–A-primer-on-reproducible-research
ORCID: http://orcid.org/0000-0001-7879-4531
Licensing: